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CLAIMS 



What is claimed is: 

1 . An apparatus for a voice user interface with 
personality, the apparatus comprising: 

logic that provides a voice user interface, 
the voice user interface outputting first voice 
signals, and the voice user interface recognizing 
speech signals; and 

logic that provides a personality, the logic 
that provides the personality interfacing with the 
logic that provides the voice user interface to 
provide the voice user interface with personality. 



the logic that provides the voice user interface and 
the logic that provides the personality comprise 
computer software stored in a memory of a computer 
20 system, the computer software being executed on a 
processor of the computer system. 



15 



2 . 



The apparatus as recited in Claim 1 wherein 
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3 . The apparatus as recited in Claim 2 wherein 
the computer system comprises a microphone and speaker, 
the speech signals being received from the microphone, 
and the first voice signals being transmitted to the 

5 speaker. 

4. The apparatus as recited in Claim 2 wherein 
the computer system is in communication with a 
telephone, the speech signals being received from the 

10 telephone, and the first voice signals being 
transmitted to the telephone. 

5. The apparatus as recited in Claim 2 wherein 
the computer system comprises at least two computers. 



6. The apparatus as recited in Claim 1 wherein 
the voice user interface with personality comprises a 
virtual assistant with personality. 



15 
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7 . 



The apparatus as recited in Claim 6 wherein 



the personality of the virtual assistant comprises a 



predetermined tone or cadence of the first voice 



signals, a predetermined vocabulary, a predetermined 
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sentence construction, or a predetermined degree of 
assertiveness . 

8. The apparatus as recited in Claim 6 wherein 
the first voice signals comprise a prompt, the prompt 
being scripted for the personality of the virtual 
assistant . 

9. The apparatus as recited in Claim 6 wherein 
the first voice signals comprise a recorded prompt, the 
recorded prompt being acted and recorded for the 
personality of the virtual assistant. 

10 . The apparatus as recited in Claim 1 wherein 
the logic that provides the personality comprises 
randomly selecting a prompt from multiple prompts 
available to provide a predetermined response. 

11. The apparatus as recited in Claim 1 wherein 
the logic that provides the personality comprises 
selecting a prompt, the prompt comprising an 
appropriate temporal prompt. 




M-5273 US 
41053S v6 



12. The apparatus as recited in Claim 1 wherein 
the logic that provides the personality comprises 
selecting a prompt, the prompt comprising a term that 

5 was previously spoken by the user in a recognized 
command . 

13 . The apparatus as recited in Claim 1 wherein 
the logic that provides the voice user interface 

10 comprises recognizing multiple spoken commands as a 
predetermined command. 

14 . The apparatus as recited in Claim 1 wherein 
the logic that provides the voice user interface 

15 comprises generating second voice signals, the second 

voice signals comprising synthesized voice signals that 
correspond to text . 

15. The apparatus as recited in Claim 1 wherein 
20 the logic that provides the personality selects a 

smooth hand-off prompt to provide a smooth hand-off 
between the first voice signals and second voice 
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signals, the second voice signals being output by the 
logic that provides the voice user interface. 

16. The apparatus as recited in Claim 15 wherein 
5 the logic that provides the personality selects a first 
complete sentence for output by the first voice signals 
and a second complete sentence for output by the second 
voice signals. 

10 17. The apparatus as recited in Claim 16 wherein 

the first voice signals comprise the voice of a virtual 
assistant, and the second voice signals comprise the 
voice of a helper to the virtual assistant. 

15 18. The apparatus as recited in Claim 17 wherein 

the second voice signals comprise recorded voice 
signals . 



20 the second voice signals comprise synthesized voice 
signals . 



19. 



The apparatus as recited in Claim 17 wherein 
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polite and socially appropriate while interacting with 
the voice user interface with personality. 

24 . The apparatus as recited in Claim 1 wherein 
5 the logic that provides the personality comprises 

controlling the voice user interface in situations in 
which negative comments are needed. 



10 the logic that provides the personality comprises 
selecting a negative prompt, the negative prompt 
comprising a concise and plain statement of a problem 
without blaming a user, and outputting a recording of 
the brief negative prompt at a lower volume. 



26. The apparatus as recited in Claim 1 wherein 
the logic that provides the personality comprises 
selecting a prompt based on a user's experience with 
using the voice user interface during a current session 

20 and across sessions. 

27. The apparatus as recited in Claim 26 wherein 
the logic that provides the personality comprises 



25. 



The apparatus as recited in Claim 24 wherein 



15 




M-S273 US 
410536 v6 

selecting a shorter prompt based on the user's 
experience with using the voice user interface during 
the current session and across sessions. 

5 28. The apparatus as recited in Claim 26 wherein 

the logic that provides the personality comprises 
selecting a longer help prompt if the user's input 
indicates a problem with increasing frequency during 
the current session and across sessions. 



29. The apparatus as recited in Claim 26 wherein 
the logic that provides the personality comprises 
selecting a prompt that provides the available options 
to the user if the voice user interface does not 

15 recognize a command spoken by the user or if the user 
has not spoken for a defined period of time. 

30. The apparatus as recited in Claim 26 wherein 
the logic that provides the personality comprises 

20 selecting a coaching prompt that provides a current 

state of interaction, provides commands that the user 
can say at the current state of interaction, and 



10 
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provides the actions that would be taken in response to 
each of the commands . 

31. The apparatus as recited in Claim 1 wherein 
5 the logic that provides the personality comprises 

selecting an approximation prompt. 

32. The apparatus as recited in Claim 1 wherein 
the logic that provides the personality comprises 

10 providing the voice user interface with a first 
personality and a second personality. 

33. The apparatus as recited in Claim 3 2 wherein 
the first voice signals comprise the voice of the first 

15 personality, and second voice signals comprise the 
voice of the second personality. 

34. The apparatus as recited in Claim 32 wherein 
a first virtual assistant comprises the first 

20 personality, and a second virtual assistant comprises 
the second personality. 
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35. The apparatus as recited in Claim 34 wherein 
the first virtual assistant and the second virtual 
assistant comprise different genders. 

5 36. The apparatus as recited in Claim 34 wherein 

a user selects the first virtual assistant or the 
second virtual assistant based on descriptions of the 
first virtual assistant and the second virtual 
assistant or based on interacting with the first 
10 virtual assistant and the second virtual assistant. 

37. The apparatus as recited in Claim 1 wherein 
the logic that provides the personality comprises 
selecting a humorous prompt. 



38. The apparatus as recited in Claim 1 wherein 
the voice user interface with personality comprises a 
virtual assistant for a voice-based desktop 
environment . 



39. The apparatus as recited in Claim 3 8 wherein 
the desktop environment comprises multiple objects, the 



15 
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virtual assistant being navigated among the multiple 
objects by a user. 

40. The apparatus as recited in Claim 1 further 
comprising: 

a recognition grammar stored in a memory, the 
recognition grammar comprising multiple phrases 
that a virtual assistant with a personality can 
recognize when spoken by a user, and the 
recognition grammar being selected based on the 
personality of the virtual assistant. 

41. The apparatus as recited in Claim 2 wherein 
the computer system comprises : 

a telephone line card in communication with a 
telephone line for receiving speech input data 
from a user and speech output data from the logic 
that provides the voice user interface. 

42. The apparatus as recited in Claim 41 wherein 
the computer system further comprises: 

a network card, the network card in 
communication with a network. 
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43 . The apparatus as recited in Claim 1 wherein 
the logic that provides the voice user interface 
comprises : 

5 echo cancellation software, barge- in 

software, signal processing software, automatic 
speech recognition/natural language software, 
request for services software, and text -to- 
speech/recorded speech software. 

10 

44 . A method for a voice user interface with 
personality, the method comprising: 

executing a voice user interface, the voice 
user interface outputting first voice signals, the 
15 voice user interface recognizing speech signals; 

and 

controlling the voice user interface to 
provide the voice user interface with a 
personality. 

20 

45. The method as recited in Claim 44 wherein the 
voice user interface with personality comprises 
computer software stored in a memory of a computer 
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system, the computer software being executed on a 
processor of the computer system. 

46. The method as recited in Claim 44 wherein the 
voice user interface with personality comprises a 
virtual assistant with personality. 

47. The method as recited in Claim 46 wherein the 
personality of the virtual assistant comprises a 
predetermined tone or cadence, a predetermined 
vocabulary, a predetermined sentence construction, or a 
predetermined degree of assert iveness . 

48. The method as recited in Claim 44 wherein the 
first voice signals comprise a prompt, the prompt being 
scripted for the personality of the voice user 
interface . 

49. The method as recited in Claim 44 wherein the 
first voice signals comprise a recorded prompt, the 
recorded prompt being acted and recorded for the 
personality of the voice user interface. 
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50. The method as recited in Claim 44 wherein the 
controlling the voice user interface comprises randomly 
selecting a specific prompt that corresponds to a 

5 generic prompt. 

51. The method as recited in Claim 44 wherein the 
controlling the voice user interface comprises 
selecting a prompt, the prompt comprising an 

10 appropriate temporal prompt. 

52. The method as recited in Claim 44 wherein the 
controlling the voice user interface comprises 
selecting a prompt to respond to a user, the prompt 

15 comprising a term that was previously spoken by the 
user in a recognized command. 

53. The method as recited in Claim 44 wherein the 
voice user interface comprises recognizing multiple 

20 spoken commands as a predetermined command. 

54. The method as recited in Claim 44 wherein the 
voice user interface comprises generating second voice 
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signals, the second voice signals comprising 
synthesized voice signals that correspond to text. 

55. The method as recited in Claim 44 wherein the 
5 controlling the voice user interface comprises 

selecting a smooth hand-off prompt that provides a 
smooth hand-off between the first voice signals and 
second voice signals, the second voice signals being 
output by the voice user interface. 



56. The method as recited in Claim 55 wherein the 
controlling the voice user interface comprises 
selecting a first complete sentence for output by the 
first voice signals and a second complete sentence for 

15 output by the second voice signals. 

57. The method as recited in Claim 56 wherein the 
first voice signals comprise the voice of a virtual 
assistant and the second voice signals comprise the 

20 voice of a helper to the virtual assistant. 

58. The method as recited in Claim 57 wherein the 
second voice signals comprise recorded voice signals. 



10 
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59. The method as recited in Claim 57 wherein the 
second voice signals comprise synthesized voice 
signals . 



60. The method as recited in Claim 44 wherein the 
controlling the voice user interface comprises 
controlling the selection of the first voice signals 
output by the voice user interface such that the voice 
10 user interface behaves consistently with social and 
emotional norms, including politeness, while 
interacting with the user. 

61. The method as recited in Claim 60 wherein the 
15 controlling the voice user interface comprises 

selecting a word that is within a recognition grammar 
to be output by the first voice signals. 

62. The method as recited in Claim 60 wherein the 
20 controlling the voice user interface comprises 

controlling the selection of the first voice signals 
output by the voice user interface such that the voice 



5 
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user interface addresses a user by a proper name of the 



63. The method as recited in Claim 60 wherein the 
speech signals comprise user commands spoken politely. 

64. The method as recited in Claim 44 wherein the 
controlling the voice user interface comprises 
controlling the voice user interface in situations in 
which negative comments are needed. 

65. The method as recited in Claim 64 wherein the 
controlling the voice user interface comprises 
selecting a negative prompt, and outputting a recording 
of the negative prompt at a lower volume. 

66. The method as recited in Claim 44 wherein the 
controlling the voice user interface comprises 
selecting a prompt based on a user's experience with 
using the voice user interface during a current session 
and across sessions. 



user . 




M-S273 OS 
410S36 v6 

67. The method as recited in Claim 66 wherein the 
controlling the voice user interface comprises 
selecting a shorter prompt based on the user's 
experience with using the voice user interface during 
the current session and across sessions. 

68. The method as recited in Claim 66 wherein the 
controlling the voice user interface comprises 
selecting a longer help propapt if the user's input 
indicates a problem with increasing frequency during 
the current session and across sessions. 

69. The method as recited in Claim 66 wherein the 
controlling the voice user interface comprises 
selecting a prompt that lists the available options to 
the user if the voice user interface does not recognize 
a command spoken by the user or if the user has not 
spoken for a defined period of time. 

70. The method as recited in Claim 66 wherein the 
controlling the voice user interface comprises 
selecting a coaching prompt that provides a current ' 
state of interaction, lists commands that the user can 
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say at the current state of interaction, and lists the 
actions that would be taken in response to each of the 
commands . 

71. The method as recited in Claim 44 wherein the 
controlling the voice user interface comprises 
selecting an approximation prompt. 

72. The method as recited in Claim 44 wherein the 
controlling the voice user interface comprises 
providing the voice user interface with a first 
personality and a second personality. 

73 . The method as recited in Claim 72 wherein the 
first voice signals comprise the voice of the first 
personality, and second voice signals comprise the 
voice of the second personality. 

74 . The method as recited in Claim 72 wherein a 
first virtual assistant comprises the first 
personality, and a second virtual assistant comprises 
the second personality. 
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75. The method as recited in Claim 74 wherein the 
first virtual assistant and the second virtual 
assistant comprise different genders. 

76. The method as recited in Claim 74 wherein a 
user selects the first virtual assistant or the second 
virtual assistant based on descriptions of the first 
virtual assistant and the second virtual assistant or 
based on interacting with the first virtual assistant 
and the second virtual assistant. 

77. The method as recited in Claim 44 wherein the 
voice user interface with personality comprises 
selecting a humorous prompt. 

78. The method as recited in Claim 44 wherein the 
voice user interface with personality comprises a 
virtual assistant for a voice-based desktop 
environment . 

79. The method as recited in Claim 78 wherein 'the 
desktop environment comprises multiple objects, the 



M-5273 US 
410536 v6 



virtual assistant being navigated among the multiple 
objects by a user. 

80. The method as recited in Claim 44 further 
5 comprising: 

storing a recognition grammar in a memory, 
the recognition grammar comprising multiple 
phrases that the voice user interface can 
recognize when spoken by a user, and the grammar 
10 being selected based on the personality of the 

voice user interface . 



81. The method as recited in Claim 44 wherein the 
voice user interface comprises: 
15 echo cancellation software, barge- in 

software, signal processing software, automatic 
speech recognition/natural language software, 
request for services software, and text-to- 
speech/ recorded speech software . 

20 

82. A data signal in a carrier wave for a voice 
user interface with personality, the data signal in a 
carrier wave comprising: 
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first voice signals, the first voice signals 
being output by a voice user interface with 
personality; and 

speech signals, the voice user interface with 
personality recognizing the speech signals. 

83 . The data signal in a carrier wave as recited 
in Claim 82 wherein the first voice signals comprise a 
prompt, the prompt being scripted for the personality 
of the voice user interface. 

84. The data signal in a carrier wave as recited 
in Claim 82 wherein the first voice signals comprise a 
recorded prompt, the recorded prompt being acted and 
recorded for the personality of the voice user 
interface . 

85. The data signal in a carrier wave as recited 
in Claim 82 wherein the first voice signals comprise a 
prompt randomly selected from multiple prompts 
available to provide a predetermined response. 
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86. The data signal in a carrier wave as recited 
in Claim 82 wherein the first voice signals comprise a 
prompt, the prompt comprising an appropriate temporal 
prompt . 

87. The data signal in a carrier wave as recited 
in Claim 82 wherein the first voice signals comprise a 
prompt to respond to a user, the prompt comprising a 
term that was previously spoken by the user in a 
recognized command. 

88. The data signal in a carrier wave as recited 
in Claim 82 wherein the speech signals comprise a 
phrase, the phrase being recognized by the voice user 
interface with personality as a predetermined command. 

89. The data signal in a carrier wave as recited 
in Claim 82 further comprising: 

second voice signals, the second voice 
signals being output by the voice user interface 
with personality. 



M-S273 OS 
410536 V6 

90. The data signal in a carrier wave as recited 
in Claim 89 further comprising: 

third voice signals, the third voice signals 
being output by the voice user interface with 
5 personality, the third voice signals comprising a 

smooth hand-off between the first voice signals 
and the second voice signals. 

91. The data signal in a carrier wave as recited 
10 in Claim 89 wherein the first voice signals comprise a 

first complete sentence and the second voice signals 
comprise a second complete sentence. 

92 . The data signal in a carrier wave as recited 
15 in Claim 8 9 wherein the first voice signals comprise 

the voice of a virtual assistant,' and the second voice 
signals comprise the voice of a helper to the virtual 
assistant . 

20 93. The data signal in a carrier wave as recited 

in Claim 89 wherein the second voice signals comprise 
synthesized voice signals. 
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94 . The data signal in a carrier wave as recited 
in Claim 82 wherein the first voice signals comprise a 
word that is within a recognition grammar of the voice 
user interface with personality. 

95. The data signal in a carrier wave as recited 
in Claim 82 wherein the first voice signals comprise a 
proper name of the user. 

96 . The data signal in a carrier wave as recited 
in Claim 82 wherein the first voice signals comprise a 
negative prompt, the negative prompt comprising a 
concise and plain statement of a problem without 
blaming a user, and the first voice signals being 
output at a lower volume . 

97. The data signal in a carrier wave as recited 
in Claim 82 wherein the first voice signals comprise a 
prompt that is selected based on a user's experience 
with using the voice user interface with personality 
during a current session and across sessions. 
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98. The data signal in a carrier wave as recited 
in Claim 97 wherein the first voice signals comprise a 
longer help prompt if the user's input indicates a 

5 problem with increasing frequency during the current 
session and across sessions. 

99. The data signal in a carrier wave as recited 
in Claim 82 wherein the first voice signals comprise a 

10 coaching prompt that provides a current state of 

interaction, lists commands that the user can say at 
the current state of interaction, and lists the actions 
that would be taken in response to each of the 
commands . 



100. The data signal in a carrier wave as recited 
in Claim 82 wherein the first voice signals comprise an 
approximation prompt. 



in Claim 82 wherein the voice user interface with 
personality comprises a first personality and a second 
personality . 



15 



20 



101. The data signal in a carrier wave as recited 
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102. The data signal in a carrier wave as recited 
in Claim 101 wherein the first voice signals comprise 
the voice of the first personality, and second voice 
signals comprise the voice of the second personality, 
the second voice signals being output by. the voice user 
interface with personality. 

103. The data signal in a carrier wave .as recited 
in Claim 101 wherein a first virtual assistant 
comprises the first personality, and a second virtual 
assistant comprises the second personality. 

104. The data signal in a carrier wave as recited 
in Claim 82 wherein the first voice signals comprise a 
humorous prompt . 



